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(57) Abstract: Handwriting data is 
received electronically from a remote 
user at a handwritten -information 
server, and the handwriting data 
is processed in accordance with 
instructions provided to the server 
by the user, e.g. by forwarding 
the handwriting data to a specified 
destination identified by the user. 
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USING HANDWRITTEN INFORMATION 
BACKGROUND 

This application claims the benefit of the filing dates of United 
States Provisional Patent Application 60/195,491, filed April 10, 
5 2000, and United States Patent Application 09/698,471, filed 
October 27, 2000, both incorporated here by reference. This 
application is a continuation in part of United States Patent 
Application 09/698,471, filed October 27, 2000. 

This invention relates to using handwritten information. 

10 A variety of devices have been used to capture handwritten 
information, including clay tablets and styli, pencil and paper, 
digitizing pads, and electronic pens. 

Personal digital assistants, such as PalmPilot, also use handwriting 
as a mode of input. 

15 SUMMARY 

In general, in one aspect, the invention features a method that 
includes receiving handwriting data electronically from a remote 
user at a handwritten-information server, and processing the 
handwriting data in accordance with instructions provided to the 
20 server by the user. 

Implementations of the invention may include one or more of the 
following features. The handwriting data is generated using a 
handwriting device (e.g., an electronic wireless pen) at the location 
of the remote user. Handwriting recognition is performed at the 
25 site of the remote user or at the handwritten-information server. At 
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the location of the remote user, an electronic file is formed 
representing the handwritten information. The handwritten 
information is transmitted wirelessly to a communication device. 
The electronically captured handwriting is then transmitted from 
5 the communication device to the handwritten-information server. 

The handwriting data includes information identifying a 
destination of the handwriting data. The processing of the 
handwriting data includes forwarding it to a destination, for 
example, by sending the handwriting data in FAX format or as an 
1 0 email attachment or in a body of an email. 

In general, in another aspect, the invention features a method that 
includes (a) using a tracker attached to a mobile personal digital 
assistant having wireless communication capability to track 
handwriting done on a writing surface in the vicinity of the 
15 personal digital assistant, (b) sending handwriting data 

representing the handwriting wirelessly from the personal digital 
assistant to a server, and (c) providing services to a user from the 
server with respect to the handwriting data. 

In general, in another aspect, the invention features a method that 
20 includes (a) storing, in a server, handwriting information provided 
wirelessly from mobile devices, and (b) communicating the 
handwriting to destinations based on instructions provided by 
users. 
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In general, in another aspect, the invention features a method that 
includes storing in a server handwriting information received 
wirelessly from mobile devices, and providing an interactive user 
interface on a screen of a mobile device to enable a user to control 
5 functions applied to the handwriting information. 

In general, in another aspect, the invention features a method that 
includes (a) capturing handwriting of a name of an addressee of a 
message as handwriting data representing motion that occurred 
during the handwriting, (b) communicating the handwriting data to 
10 a server, and (c) at the server, determining the name from the 
handwriting data using information stored in a database. 

Implementations of the invention may include one or more of the 
following features. The user is asked for confirmation of the 
located address. The message is sent to the located address. The 
15 database includes information provided by a user, the information 
in the database linking names with corresponding addresses. 

In general, in another aspect, the invention features a method that 
includes from a server, communicating handwriting data to remote 
user devices, the handwriting data for each of the devices being 
20 formatted according to display capabilities of the remote user 
device. 

In general, in another aspect, the invention features a method that 
includes (a) receiving at a server handwriting data representing a 
signature of a user of a remote device, (b) verifying the signature 
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based on handwriting data associated with the user and stored on 
the server, and (c) authorizing a transaction based on the 
verification of the signature. 

Implementations of the invention may include one or more of the 
5 following features. The authorizing includes providing an 

authorization to a third party. The transaction is associated with a 
financial account of the user. The receipt of the handwriting data 
occurs after terms of the transaction have been resolved. The 
authorization is also based on a unique identifier of a pen that is 
10 used to write the signature. The transaction may be authorized 
based also on voice or speech recognition and on a time stamp. 

In general, in another aspect, the invention features a method that 
includes (a) receiving and storing at a server handwriting 
information that is created by users at remote locations and 
1 5 communicated electronically to the server, (b) electronically 

performing services with respect to the stored information, and (c) 
charging the users for the services performed. 

In general, in another aspect, the invention features a method that 
includes (a) receiving, at a server, handwriting data from a pen and 

20 a pen identifier, (b) storing, at a server, an indication that the pen is 
in the possession of a person not authorized to use it, (c) in 
response to the received handwriting data and pen identifier, 
determining that the pen is not being used by an authorized person, 
and (d) taking an action in accordance with the result of the 

25 matching. 
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In general, in another aspect, the invention features, a method that 
includes (a) receiving electronic handwriting data created by a 
user's handwriting, (b) receiving instructions (e.g., from a 
handwriting instrument) indicating visible qualities (e.g., color) to 
5 be associated with respective portions of the handwriting, and (c) 
manifesting the handwriting in a visible form in which each of the 
respective portions of the handwriting is manifested in accordance 
with the indicated visible qualities. 

In another aspect, the invention features a method that includes (a) 
1 0 associating unique identifiers with respective electronic 

handwriting devices that are useful in generating handwriting data, 
(b) in a server, storing the handwriting data and storing 
associations between the unique identifiers of the devices and 
registered unique identifiers of respective users, and (c) at the 
1 5 server, controlling use of the handwriting data based on matching 
stored pairs of user identifiers and device identifiers to confirm that 
the purported user of a device used to generate the handwriting 
data is the registered user of the device. 

In general, in another aspect, the invention features apparatus that 
20 comprises a mobile electronic device that includes (a) a 

communication facility configured to communicate with a remote 
location, and (b) a tracking facility configured to track handwriting 
motion of a stylus on a surface in a vicinity of the mobile 
electronic device. 
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Implementations of the invention may include one or more of the 
following features. The mobile electronic device may include a 
mobile telephone. The mobile electronic device has a housing and 
the tracking facility is incorporated within the housing or is 
5 attached to the housing. The device has a receptacle for a stylus. 

Other advantages and features will become apparent from the 
following description and from the claims. 

DESCRIPTION 

(Figure 1 shows a block diagram of a handwritten information 
10 processing system. 

Figures 2 through 4 show word formats for handwriting data. 

Figure 5 shows a user interface. 

Figures 6 through 1 1 show user interface screens on a cell phone. 
Figure 12 shows a cell-phone and pen. 
15 Figure 13 shows an authentication system. 
Figure 14 shows six sides of a cell phone. 

Figure 15 shows six steps in an example use of a pen-equipped cell 
phone.) 

As shown in Figure 1, handwritten information (such as notes and 
20 pictures) 10 produced by a user using a pen 12 is converted to 
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handwriting data 1 1 and sent through a cellular phone 13 and the 
Internet 14 to a handwritten-information server 16. 

The pen could be any device by which the motion of a user's hand 
can be captured as data, including electronic pens, digitizing 
5 tablets, mouses, and trackballs, to name some. The pen could be of 
the kind that can be tracked by nearby tracking devices such as the 
pens described in United States Patent Application Serial Numbers 
60/096,988, filed August 18, 1998; 60/142,200, filed July 1, 1999; 
and 60/142,201, filed July 1, 1999, and 09/698,471, filed October 
10 27, 2000, all incorporated by reference. The tracking device can be 
housed in a cap of the pen, in a pen well, or in (or on) the cell 
phone or similar device. We use the word pen broadly to refer to 
any of these. 

The handwriting data (representing the handwriting motion of the 
15 pen) can be encoded in a variety of formats. The encoding can 

include data compression. For purposes of viewing the handwriting 
later through a web browser, a free browser plug-in, capable of 
decoding the encoded data, can be downloaded from a server. 

Handwriting and handwritten information may include written 
20 letters, text, words, notes, numbers, sketches, and any other indicia 
that can be formed by motion of a user f s hand. 

The cell phone could be replaced by any similar kind of device that 
permits communication with a remote server, either wirelessly or 
through a wired connection. For example, the device could be any 
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wireless Internet-ready device such as a PalmPilot or other 
personal digital assistant, a laptop or desktop computer, or a wired 
telephone set. Dial-up devices could reach the server through a 
phone or a modem. Any Internet-ready wireless or dial-up device 
5 that has limited ability to capture and convey handwriting data can 
be supplemented by a wireless portable electronic pen, pad or 
scanner that can beam handwriting data to the Internet-ready 
device. 

When the device 13 communicates with the server, the handwriting 
10 data 1 1 (the content) that was sent from the pen (or the tracker, 
depending on the implementation) is supplemented with a sender 
ID 22 (also typically sent from the pen), an identifier for an 
intended type 14 of delivery or method of communication (e.g., 
fax, e-mail, paging, or numerical message) 24, and a destination 
15 (e.g., IP) address 21. 

The desired destination address (such as a telephone number) may 
be entered at the cell phone keypad or may be provided from an 
address book 25 maintained in the pen or in the cell phone, 
allowing the user to select a destination address or fax number, for 
20 example. The destination address may also simply be handwritten 
into the message for later extraction. 

The server 16 processes the handwriting data 1 1, extracts the 
destination address 21 (using handwriting recognition, if the 
address has been handwritten) and converts the handwriting data to 
25 a fax message 30 or to an e-mail attachment 32, or uses 
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handwriting recognition to send a message as an ASCII e-mail, or 
generates a paging message 34 or a numerical message 36. 

The handwritten-information server provides a wide variety of 
services for the user with respect to the handwriting data. One 
5 service is to process the handwriting data and then forward it 1 8 by 
email or FAX to a destination 20 the address 21 of which has been 
specified by the user. The user may be charged 38 for the delivery. 

In some implementations, the connection to the handwritten- 
information server is originated by the cell phone, which also 
10 provides the destination addresses and ID information about the 
user. In other implementations, the pen itself originates the 
connection and provides the destination address and ID. 

In the first of these two modes, the user works in a browser 40 
running on the cellular phone to connect through the Internet 14 to 
15 the handwritten-information server 16. The browser could be a 

standard Wireless Application Protocol (WAP) browser or another 
browser. 

Through the browser, the user specifies the destination address 21 
for the handwriting data and the mode of delivery or 
20 communication in which the handwriting data should be sent (for 
example, FAX, email, voice). The user ID could also be provided 
automatically to the handwritten-information server by the user's 
wireless service provider 42 (e.g., ATT, Sprint, Air Touch) or by 
the user when he logs onto the handwritten-information server. 
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The handwritten-information server locates the user's account 44 
based on the user's ID and then retrieves a serial number 46 of the 
pen from a pre-stored list that maps user IDs to pen serial numbers. 
The list is generated from information provided by users when they 
5 register for the service or when they log on. Each pen has a unique 
serial number that is assigned, for example, by the manufacturer or 
distributor. 

The pen serial number 46 is provided to the server along with the 
handwriting data each time the pen sends data. The handwritten- 
10 information server watches for handwriting data coming from a 
■ pen that has the serial number associated with the logged on user. 

The user can "talk 11 to the server by other mechanisms than a 
browser, including speech that is recognized automatically at the 
server end or touch-tone (DTMF) sequences. 

1 5 Once the user's browser has setup a planned transfer with the 
handwritten-information server, the pen can connect through a 
serial or infrared interface 52 of the cellular phone to the 
handwritten-information server (or, if the tracker is in or on the 
phone, through a wired connection within the phone) and submit 

20 the handwriting data. For this purpose, the server's address 33 (IP 
or DNS) is pre-stored in the pen's memory as part of the 
configuration settings. 

In implementations in which the tracker is not part of or attached to 
the phone, the pen 12 has a beaming capability that enables it to 
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convey the handwriting data to the cell phone using an infrared 
link (IrDA), an RF link (bluetooth) or a similar mode of 
communication. Alternatively the communication could occur 
directly between the pen/tracker and a remote server using a 
5 wireless connection to the Internet, such as a connection that 

conforms to the IEEE 802.11 protocol. In that case, the pen/tracker 
could be supplied with an 802.1 1 -compliant chip. 

If the service provider 42 that serves the cell phone does not 
provide Internet data connectivity to the handwritten-information 

10 server, the pen dials (through the cell phone) any Internet Service 
Provider (ISP) and opens a socket connection to the handwritten- 
information server, or dials the handwritten-information server 
directly. In the case of an Internet connection, the pen uses SLIP or 
PPP protocol to transfer the handwriting data through the cell 

1 5 phone. In the case of a direct connection to the handwritten- 
information server, the communication can be done using any 
protocol negotiated between the pen and the server. 

After receiving the handwriting data, the handwritten-information 
server preprocesses the data, including reconstructing the 

20 handwriting from the angle/position sensor data, and converts the 
handwriting data to the appropriate format (e.g., FAX, email 
attachment) as specified by the user and sends it to the specified 
destination in FAX, email attachment, or other format. In other 
examples, the handwriting can be included directly as handwriting 

25 in the body of an email. 
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In some implementations, the handwriting data is processed 
without regard to the "content" of the handwriting and forwarded 
to the destination. In other implementations, some or all of the 
content of the handwritten information may be extracted using 
5 handwriting recognition algorithms. The recognition can be 
performed at the pen, at the cell phone, or at the handwritten 
information server. 

The recognized content could include, for example, the words of a 
handwritten message. In that case, the handwritten-information 
1 0 server may send the text of the message in the body of an email or 
in a synthesized voice message, or may store the message in 
character format for later use. The content could be written using 
standard alphanumeric characters or special machine-readable 
signs. 

15 The recognized content could also include the destination address 
in the form of a FAX number or an e-mail address written by the 
user on paper. In that case, it is possible to sidestep the need for a 
browser on the cell phone and enable the pen to communicate the 
handwriting data directly through the cell phone to the 

20 handwritten-information server. The same thing could be done 

without the need for handwriting recognition using a user interface 
display or buttons on the pen. 

In some cases, the destination address may be derived from the 
written information without requiring time-consuming handwriting 
25 recognition of the content of the handwriting. Instead, the 
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handwriting can simply be matched with pre-stored handwritten 
versions of addressees, a process that is faster and simpler. 

In addition to forwarding handwriting data (representing, e.g., 
notes, drawings, text) as e-mail attachments or FAX, the 
5 handwritten-information server can provide a range of other 
products and services. 

The handwritten-information server can assist pen users in other 
aspects of communication. For example, the handwriting data 
received from the pen may be stored in user mail boxes 48 

10 maintained on the handwritten-information server for later retrieval 
or forwarding. The handwriting data may be posted on the Internet 
14. The handwritten-information server may store information 
such as addresses, phone numbers, or to do lists in a user's 
personal space 60. The pen may also request the handwritten- 

1 5 information server to provide information on the user's account 44, 
perform computations that on-pen processors cannot do, and set or 
re-set preferences on the pen or on the handwritten-information 
server. 

The system can be operated in a model in which the electronic pen 
20 is provided free or nearly free to users. 

A more detailed implementation discussion follows. 

Once the pen is in communication with the cell phone, the 
handwriting data content of the pen's memory 64 is automatically 
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transmitted to the server 16. The server receives the compressed 
data content 62 and the header information (sender ID and delivery 
type) from the pen and stores them in its database 66. 

The server then updates the user account (by storing the received 
5 data and the delivery type) and waits for further instructions that 
will come from the cell phone or a PC-based Internet connection 
on what to do with the received data. The server may send an 
acknowledgement of receipt to the pen and may cause the pen to 
erase the message according to preferences defined by the user, for 
10 example, a preference to erase when sent or to erase within three 
days. 

* The pen captures the handwriting data 11 as x, y vectors. In some 
cases the x, y vectors could be supplemented by additional data 
representing pen pressure (useful for determining line thickness), 

1 5 pen tilt (for accuracy), and virtual color. A virtual color or other 
handwriting characteristic can be indicated by the user through a 
button click on the pen or by writing an appropriate codeword. 
Thus, the user could indicate that the first sentence in memo should 
be construed to be in red and the second sentence in blue, even 

20 though the actual ink color of the pen is black. When the 

handwriting is later manisfested to a viewer, say on a display of a 
handheld device, each portion of the handwriting (or the derived 
characters) can be displayed in the indicated color. 

As shown in figure 2, each of the vectors is captured as a two-byte 
25 word. Fifteen bits of each byte represent one of the coordinate 
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values in a two-coordinate system. The coordinate values are either 
real paper coordinates with .1 mm resolution or are sensor sub- 
pixel coordinates. The maximum value for a paper coordinate is 
3000. The maximum value for a sub-pixel coordinate is 1024 times 
5 a scale factor between 4 and 16. 

The successive coordinates for handwriting on a page are captured 
in successive frames and stored together. A timestamp is inserted 
in the beginning of the page and each time the pen is lifted from 
the paper. The system detects when the pen is lifted or again 

10 contacts the paper by detecting the presence or absence of the pen 
in successive frames. The timestamp can be kept in either a full 
format or an incremental format at shown in figures 3 and 4. In 
either case the most significant bit distinguishes the time stamp 
word from a vector word. When a new page is started, a full 

15 timestamp is inserted before the first valid pixel word. Subsequent 
timestamps are incremental. An incremental timestamp stores only 
the number of frames since the last full timestamp. The frame 
frequency is a settable parameter. 

A full timestamp can also be inserted anywhere within a page. 
20 Incremental timestamps will show incremental counter relative to 
the last full timestamp. 

Each page is stored in a separate file. There is no need for an end 
of page mark. A previous page is compressed and stored before 
new page acquisition is started. 



15 



WO 01/77796 



PCT/US01/11486 



After capture, the vector data is pre-processed and compressed 
using a smoothing and variable length coding algorithm. The 
compressed data is stored in the pen's local memory 64 until the 
pen is connected to a cell phone. When that occurs, the server 
5 address 21 and compressed data 62 are transmitted. If transmission 
is successful, the content in the pen's memory can be deleted. If 
transmission fails, a warning light on the pen indicates the failure. 

After the server receives data from the pen, the server 
decompresses the data and reads the header information, which 
10 includes the pen serial number and the user ID. The server checks 
if the ID is valid. If so, the server stores the decompressed data in 
the user's mailbox 48 or other area in the database. The stored 
decompressed files are then available to the user. 

The user may connect to the server at any time using a WAP phone 
15 or a PC with a World-Wide Web Internet connection. Once 

connected, the user can manage his account and his mailbox and 
arrange for any desired use of the handwriting data that he has 
stored. 

For example, if the user connects using a WAP phone, the user 
20 may instruct the server to perform one of the following actions: 

1 . E-mail any or all of the files to one or more specified e- 
mail addresses. The file can be sent as an attachment (or, as 
previously mentioned, directly as handwriting in the body of the 
email) in either jpeg or another format. 
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2. Fax any or all of the files to one or more specified fax 
numbers. 

3, Delete or rename one or more of the files. 

4, Perform handwriting recognition on one or more of the 
5 files. 

5. View one or more of the files. 

For example, when a user connects to his account, the server 
verifies the user's password and responds with a message such as: 

Last upload on 7/10/2000 with 6 
10 files/pages. Select appropriate 

option : 

1. send e-mail 

2 . send fax 

3. delete 

15 4 . rename 

When option 1, 2, 3, or 4 is selected, the server asks the user to 
select the files, for example if 1 is pressed, the reply message may 
be: 

Which files/pages to e-mail? 
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Press 0 to e-mail all pages or 
select a page number followed by a 
pound sign as many times as 
needed. 

5 Press pound sign one more time to 

complete or star to cancel. 

After specifying which pages, the user specifies a recipient's fax 
number or e-mail address in response to a message that reads: 

Please input destination e-mail 
10 address using keys or the 

bookmark. 

The user dials a fax number or punches an e-mail address using 
buttons on the cell phone or using pre-stored numbers or 
bookmarks on the cell phone. 

15 To make use of the features available on the server, the first time a 
user connects, he is required to register for an account and provide 
the following information: 

user name 
user password 
20 pen serial number (from the product box 

or CD-ROM) 
user name 
company name 
address 
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telephone 

e-mail 

fax 

cell phone number 
5 phone type (pull up menu) 

phone service provider (pull up menu) 



If the pen serial number is determined to be valid (because it 
matches a pen serial number previously stored at the server in 
10 connection with the sale and distribution of the pen), the user is 
registered. 

For each new user, the server (1) creates an account 44 and 
associates it with the pen serial number and the user ID and (2) sets 
up serial connection parameters to handle pen-to-cell phone 
15 connections based on the specific type of cell phone and cell phone 
provider the customer is using. 

The server can be used as a central location to store much or all of 
a user's handwritten notes and information. Because the user has 
access to the handwriting data from anywhere at anytime, he can 
20 use handwritten notes in a wide range of new ways, treating the 
central server as a personal notebook that is accessible at any time 
from any location. 

For example, a user could store a handwritten map of a location 
where a friend should plan to meet him in an unfamiliar city. The 
25 user sends the map to the friend's cell phone using the server's 
functionality. 
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Server user interface 

As shown in figure 5, the main user's screen as it might appear on a 
display has three areas: on top are the icons 70 that perform 
allowable functions on user's files, on the left are the buttons 72 
5 that provide general information, change options, and logout, and 
on the right is a list 74 of user's folders and files. 

The icons on the top are: 

View 71 —enables a user to display and perform 
editing on a handwritten file/page. 

10 E-mail 73— enables a user to email a file. 

Fax 76~enables a user to fax a file (faxing is 
through a third party arrangement that may involve 
usage charges). 

Convert 78— enables a user to have handwriting 
1 5 recognition performed. 

New Folder 80— enables a user to create a new 
folder. 

Move 82— enables a user to move a file from one 
folder to another. 

20 Rename 84— enables a user to rename a file from a 

name that is automatically applied at the time of 
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upload (e.g., <Date MonfhYear_x.scb> where x is a 
file/page number for example: 08july2000_l.scb) 

Upload 86— enables a user to send a file from a local 
disk to the server. 

5 Download 88— enables a user to send a file from the 

server to the local disk. 

Delete 90~enables a user to permanently remove a 
file from the server. 

e-mail 

10 Invoking the e-mail option requires the user to specify a 

destination e-mail address(es). The server provides an e-mail client 
67 with all necessary e-mail attributes. 

The user's email list 61 on the server could be synchronized with 
other contact lists maintained by the user. In a simpler approach, 
15 there is no synchronization. Instead, the user types e-mail 

addresses on e-mail that is originated by the user, or the file to be 
sent could be downloaded to the user's computer where his resident 
contact list 25 could be used for e-mail addressing. 

Viewing 

20 When the user triggers the view feature, a viewer 41 in the form of 
a plug-in running on the server or on the device 13 is invoked. The 
viewer allows the user to perform simple manipulation of the 
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handwritten image displayed. The manipulation includes erasing 
blocks of text, drawing free lines, allowing typed annotations, and 
selecting part of the image for cut and paste. 

Account maintenance 

5 The buttons 72 on the left of the display are: 

Your account— enables the user to review his 
account information, usage, storage size, records of 
e-mails and faxes, and records of services and to 
change or update user information, cell phone 
10 number, or cell phone provider. 

Options—enables the user to specify user options 
such as selecting an e-mail attachment default 
format (jpeg or another format), automatically 
deleting user files after so many days, and others. 

15 Help— enables the user to get help on how to use the 

server's features. 

Logout— enables the user to sign off the session 
File List 

The list of user files and folders 74 is a main part of the display 
20 that contains a hierarchical view of a user's collection of 
handwritten data in a format similar to Microsoft Windows 



22 



WO 01/77796 



PCT/US01/11486 



Explorer. The user is enabled to sort his files and folders by name 
or date. 

Viewer 

The viewer 41 is a program that allows users to open files that 
5 contain handwritten data. The handwritten data files are stored in 
the format illustrated in figures 2, 3, and 4. Compression is used to 
reduce the size and improve security of handwritten files. 

The viewer is a PC-executable file that decompresses and displays 
such files. The viewer size (in bytes) should be as small as possible 
10 because it needs to be downloaded frequently or included along 
with data files efficiently (i.e., self-executable handwritten files). 
The viewer provides the following features: 

Double clicking on a file shown on the interface of 
figure 5 launches the viewer and opens the file. 

1 5 The viewer re-constructs handwritten text in a 

scrollable window. 

The windows can be merged or split. 

The viewer Window includes a menu bar with the following 
menus: 

20 File 

Open ... 
Close ... 

Send as e-mail ... 
Save ... 
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Save as ... - save as jpg, gif 
Download - Download directly to a 
connected USB device 

Connect - connect directly with a 
5 USB device for real-time display 

Exit 

Edit (works on selected area) 
Undo 
Copy 

10 Cut 

Paste 
Delete 
Select All 

Pen (pen tool that allows users to mark 
15 the page) 

Width 
Color 

Change Color 

View 

20 Single page 

All pages 
Zoom 

Up 

Down 

25 Window 

Next window 
Arrange All 
Merge windows 
Split windows 
30 "Name of the active Window" 

In addition a tool bar can be created with most of the features 
described in the menu. 



Sending messages over the cellular phone connection 

35 There are different ways of specifying destination address and 

communication methods. The destination address can be a cellular 
phone number for sending graphical messages (using alarm 
notification capabilities of WAP), an e-mail address, or a FAX 
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number. Similarly, the communication protocol can be for a 
graphical message, an e-mail, or a FAX. 

Destination and communication data can be entered automatically 
or in a way that is natural for the user, by writing it in the top 
5 section of the page that is being sent, with manual confirmation, 
correction, and entering of data when automatic mode does not 
work. 

For example, the user may write: "Message to John Smith" at the 
beginning. The addressee can then be derived either by brute-force 
10 handwriting recognition or by a simpler matching process against 
stored samples, as mentioned earlier. 

/The user can select pen parameters (using the Viewer) to specify 
whether to transmit data to a recognized address without 
confirmation, or to require confirmation in any case. By default, 
1 5 confirmation should be required for security. 

After the handwritten message is uploaded to a server through a 
cellular phone, the text in the top section of page is parsed and 
recognized. If the destination address and communication method 
are legible and match a corresponding address and method stored 
20 on the server (for example, the address matches the address of a 

person in the address book, and the method is recognized), then the 
message is sent to the user for confirmation: "Message for John 
Smith?" 
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If confirmation is received, the message is sent to the destination 
with a confirmation to the user on the cellular phone: "SMS has 
been sent to John Smith". 

If the name or method is not recognized, the message is sent back 
5 to the user of the cellular phone for confirmation and correction 
using manual keys. The same is done if two or more people are 
found with the same name. Then the user scrolls on the cellular 
phone display and confirms the name to be used. The same is done 
if the method, SMS, e-mail or FAX is not recognized. 

10 The name that is associated with cell or FAX number or e-mail 

address or the method of communication can be entered using keys 
on the cellular phone. Manual entering of data can also be done in 
a separate connection to the server when navigation on the server 
is done using WAP protocol. 

15 Selection of type of communication, address and name of 
recipient 

Handwritten text received at the server is searched for special 
attributes such as "Message to", "E-mail to", or "FAX to". 

Text following these attributes is recognized and matched with 
20 names in the user's server-stored address book. First name, 
nickname, last name, or combinations may be used and even 
recommended to user for better recognition. 
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The second line will have a subject that will be recognized and 
kept as a name of the file for searching of messages. For example: 

Message to John Smith 

Subject Holiday Party 

5 Specific words in the document can be circled, and those words 
will be recognized and also used for search purposes. 

Messages may consist of several cell pages that are scrollable on a 
cellular phone. 

Compression for display on the cell phone screen may be limited 
10 by minimum size of letters. Horizontal scrolling may also be used, 
and text may be broken left to right as well as top to bottom. 

For example, written text appears on paper as: 

I am coming back from Miami, but had to make a stop 
at JFK in NY. 

15 I will be back in office this afternoon and talk to 
you about my ideas . 

On the phone screen, the text would be shown as: 

I am coming back from Miami, 
but had to make a stop at JFK in NY. 
20 I will be back in office this afternoon 

and talk to you about my ideas . 
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Some uses 

Messages and e-mails can be sent by specifying a name from the 
user's address book 61 stored on the server and can be 
synchronized with user's contact lists such as Microsoft Outlook. 
5 For example, if a user handwrites 

Send to: David 

the software on the server searches in the user's address book, 
matches David with David Smith, finds David Smith's e-mail 
address: dsmith@aol.com, and sends the e-mail to David. 
1 0 Handwriting recognition need only find the match on the limited 
list of names in the user's address book. 

Messages and e-mails can be formatted to suit a user's device. The 
server thus provides device independence for the data that is to be 
delivered. A message being delivered to a cell phone will be 
15 converted to a graphics format that the recipient's cell phone can 
display. 

By compressing and storing handwriting data using an efficient 
vector format the file size is kept small to reduce transmission cost. 

The server can link accurate recognition of a signature of a user 
20 with rules for transactions that are to occur based on the signature. 
For example, a server can verify the signatory and indicate to a 
third party a level of authorization associated with the recognized 
signature. For example, Joe Smith may only be authorized to 
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purchase items costing less than $500, or Jim Forbes may not 
authorize transfers from his account. 

Recognized signatures can be used as a basis for confirming, for 
example, information given in a telephone call. After the telephone 
5 conversation, the user can use the pen to confirm the subject of the 
call by sending a pen-written signature. The signature is processed 
by the server and verified (in addition to how it looks, the server 
may include the speed of writing in verification, which is difficult 
for a forger to duplicate). 

10 The signature pen could be used for remote money management by 
not permitting withdrawal or transfer money over the phone, 
except upon receipt and recognition of the handwritten signature. 

With advances in voice and video communication, completing a 
remote business transaction is feasible when the electronic 
15 signature is provided and verified, using the pen and an authorized 
server. Remote public notary functions could also be provided. 

A bank can give special bank pens to selected customers. The 
customers can then bank remotely by giving transaction 
instructions using handwriting. Pen instructions are sent to the 
20 bank, authenticated and acted on. (The combination of the unique 
pen serial number and the customer's signature and a time stamp 
provide authorization for the transaction). 
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As shown in figure 13, authentication could be done at an 
authentication server 102. using information, received from the 
phone 104, including uttered information from a speaker 106 and 
handwritten information from the pen/tracker 108. The uttered 
5 information could be recognized using either voice recognition (to 
identify the speaker by voice characteristics) or by speech 
recognition. 

A large company may issue electronic pens to its employees. Each 
pen has a unique number that identifies its user. Employees use the 

10 pens to send e-mails, messages, or requests, throughout the 

company's intranet. The meeting notes can be sent instantly to the 
intended recipients. A meeting, which used to be a decision- 
making process, becomes an action-taking process. The server 
may have lists of pen pals, departments, and top brass so sending 

15 information is easy. The user may specify, for example: "send to 
my department" and all employees in his department will get it. 

The pen and server can be used as an Internet collaboration tool. A 
user writes on paper and through the Internet his message is 
• displayed on any computer with an Internet connection. This is 
20 similar to Microsoft NetMeeting but instead of a keyboard the user 
can use a pen. 

Cell phone user interface 

By way of example, in figure 6, the introductory screen of the user 
interface that is displayed on the screen of a cell phone includes a 
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welcome line and a choice between "login" and "register". In 
figure 7, if the user has selected login, he is invited to enter his user 
id. In the example he has entered "greg". In figure 8, the password 
has been requested and entered. In figure 9, the user is welcomed 
5 and given a choice among "iribox 1 , "send email", "send last page" 
and "send last page to". In figure 10, after invoking the inbox, the 
user is shown a list of documents. Having picked the first listed 
document, in figure 11, the user is given the chance to send either 
of two stored .bmp files of handwriting, or to move up one level in 
1 0 the menu hierarchy or to move to the home page. 

Handwriting sensors in cell phone 

As shown in figure 12, the sensor unit could be attached to or 
incorporated into the cell phone, for example on the side wall of 
the phone. In this approach, the user would only need to carry the 

1 5 pen 94 and the sensor equipped cell phone in order to be able to 
produce, store, and use handwritten information. When the user 
wished to handwrite information 96 to be uploaded to the server, 
he would put the cell phone on the edge of a writing surface 98 and 
proceed with handwriting. The handwriting data would be 

20 generated in the cell phone and then sent to the server as described 
above. Within the cell phone, a single processor and memory could 
serve both the tracking function and the cell phone functions. 

Figure 14 shows six views of a pen-equipped cell phone 208. The 
pen 210 can be stored in a pen well 212 that is accessible at one 
25 end of the phone. Sensors 214, 21 6 of the tracker module can be 
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,seen on one side wall of the phone. Pads 218 on the bottom of the 
phone enable it to resist slipping on a surface when it is placed 
there for purposes of capturing handwriting. The tracking module 
can be built into the phone or can be provided as a module that 
5 plugs into the phone or is permanently built in to the phone. 

As shown in figure 15, in an illustrative use, the pen 210 is 
withdrawn from the cell phone 208. The cell phone is placed on a 
writing surface, and the pen is used to sketch a map 230. The map 
is captured as handwriting by the sensors of the embedded tracker. 
10 The cell phone then dials a number of the server. Later, at a 

different location, a cell phone 240 rings, receives the map and 
displays it 232. 

The server can provide a service that helps to retrieve a stolen or 
lost pen. The user notifies the server that the pen has been stolen. 

1 5 Later when handwriting data associated with the identified pen is 
received at the server, the server may send a reply message 
indicating that the pen has been stolen and that a reward is offered 
for its safe return. Other actions might also be taken, such as 
alerting the owner that handwriting from the lost or stolen pen has 

20 been received. 

Other implementations are also within the scope of the following 
claims. 
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CLAIMS 

1 . A method comprising 

receiving handwriting data electronically from a remote 
user at a handwritten-information server, and 

5 processing the handwriting data in accordance with 

instructions provided to the server by the user. 

2. The method of claim 1 in which the handwriting data is 
generated using a handwriting device at the location of the remote 
user. 

10 3 . The method of claim 1 in which the handwriting data is 
generated by an electronic wireless pen. 

4. The method of claim 1 also including 

performing handwriting recognition at the site of the remote user. 

l 5. The method of claim 1 also including 

1 5 performing handwriting recognition at the handwritten-information 
server. 

6. The method of claim 1 also including 

at the location of the remote user, forming an electronic file 
representing the handwritten information, 
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transmitting the handwritten information wirelessly to a 
communication device, and 

transmitting the electronically captured handwriting from the 
communication device to the handwritten-information server. 

5 7. The method of claim 1 in which the handwriting data 
includes information identifying a destination of the handwriting 
data. 

8. The method of claim 1 in which the processing of the 
handwriting data includes forwarding it to a destination. 

10 9. The method of claim 8 in which the forwarding comprises 
sending the handwriting data in FAX format. 

10. The method of claim 8 in which the forwarding comprises 
sending the handwriting data as an email attachment or in a body 
of an email. 

15 11. A method comprising 

using a tracker attached to a mobile personal digital 
assistant having wireless communication capability, tracking 
handwriting done on a writing surface in the vicinity of the 
personal digital assistant, 

20 sending handwriting data representing the handwriting 

wirelessly from the personal digital assistant to a server, and 
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providing services to a user from the server with respect to 
the handwriting data. 

12. A method comprising 

storing, in a server, handwriting information provided 
5 wirelessly from mobile devices, and 

communicating the handwriting to destinations based on 
instructions provided by users. 

13. A method comprising 

storing in a server handwriting information received 
10 wirelessly from mobile devices, and 

providing an interactive user interface on a screen of a 
mobile device to enable a user to control functions applied to the 
handwriting information. 

14. A method comprising 

1 5 capturing handwriting of a name of an addressee of a 

message as handwriting data representing motion that occurred 
during the handwriting, 

communicating the handwriting data to a server, 

at the server, determining the name from the handwriting 
20 data using information stored in a database, and 



35 



WO 01/77796 



PCT/US01/11486 



15. The method of claim 14 also including asking a user for 
confirmation of the located address. 

"16. The method of claim 14 also including sending the message 
to the located address. 

5 17. The method of claim 14 in which the database includes 
information provided by a user, the information in the database 
linking names with corresponding addresses. 

18. A method comprising 

from a server, communicating handwriting data to remote 
10 user devices, the handwriting data for each of the devices being 
formatted according to display capabilities of the remote user 
device. 

19. A method comprising 

receiving at a server handwriting data representing a 
1 5 signature of a user of a remote device, 

verifying the signature based on handwriting data 
associated with the user and stored on the server, and 

authorizing a transaction based on the verification of the 
signature. 

20 20. The method of claim 19 in which the authorizing includes 
providing an authorization to a third party. 
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21 . The method of claim 1 9 in which the transaction is 
associated with a financial account of the user. 

22. The method of claim 1 9 in which the receipt of the 
handwriting data occurs after terms of the transaction have been 

5 resolved. 

23. The method of claim 19 in which the authorization is also 
based on a unique identifier of a pen that is used to write the 
signature. 

24. The method of claim 19 also including authorizing the 
1 0 transaction based on voice or speech recognition. 

25. The method of claim 19 also including authorizing the 
transaction based on a time stamp. 

26. A method comprising 

receiving and storing at a server handwriting information 
15 that is created by users at remote locations and communicated 
electronically to the server, 

electronically performing services with respect to the stored 
information, and 

charging the users for the services performed. 

20 27. A method comprising 
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receiving, at a server, handwriting data from a pen and a 
pen identifier, 

storing, at a server, an indication that the pen is in the 
possession of a person not authorized to use it, and 

5 in response to the received handwriting data and pen 

identifier, determining that the pen is not being used by an 
authorized person, and 

taking an action in accordance with the result of the 
matching. 

10 28. A method comprising 

receiving electronic handwriting data created by a user's 
handwriting, 

receiving instructions indicating visible qualities to be 
associated with respective portions of the handwriting, and 

1 5 manifesting the handwriting in a visible form in which each 

of the respective portions of the handwriting is manifested in 
accordance with the indicated visible qualities. 

29. The method of claim 28 in which the visible qualities 
include color. 

20 30. The method of claim 28 in which the instructions are 
received from a handwriting instrument. 
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31. A method comprising 

associating unique identifiers with respective electronic 
handwriting devices that are useful in generating handwriting data, 

in a server, storing the handwriting data and storing 
5 associations between the unique identifiers of the devices and 
registered unique identifiers of respective users, and 

at the server, controlling use of the handwriting data based 
on matching stored pairs of user identifiers and device identifiers 
to confirm that the purported user of a device used to generate the 
10 handwriting data is the registered user of the device. 

32. Apparatus comprising 

a mobile electronic device that includes 

a communication facility configured to communicate with a remote 
location, and 

15 a tracking facility configured to track handwriting motion of a 
stylus on a surface in a vicinity of the mobile electronic device. 

33. The apparatus of claim 32 in which the mobile electronic 
device comprises a mobile telephone. 

34. The apparatus of claim 32 in which the mobile electronic 
20 device includes a housing and the tracking facility is incorporated 

within the housing. 
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35. The apparatus of claim 32 in which the mobile electronic 
device includes a housing and the tracking facility is attached to 
the housing. 

36. The apparatus of claim 32 in which the mobile electronic 
5 . device includes a receptacle for a stylus. 



40 



WO 01/77796 



PCT/US01/11486 



1/9 




-HANDWRITING DATA(CONTENT) 
-DESTINATION (IP) ADDRESS 
"SENDER ID 
-DELIVERY TYPE 
-PEN SERIAL NUMBER 



CHARGE TO USER 



O f INTERNET p O 
K 13 



ADDRESS BOOK / 

SERVER ADDRESS 

HANDWRITING^ 
DATA CONTENT 



CO IMPRESSED" 
DATA CONTENT 



^33 



■11 



62 



46- 



44 



25 



ADDRESS BOOK 

BROWSER 

VIEWER\-40 



■41 



66^ 



18 



\ 



20 

(ADDRESS 
21) 




NUMERICAL 
MESSAGING 



\ USER ID-PEN SERIAL NUMBER MAP 
^USER ACCOUNT AR 
USER MAILBOX^ ^61 
USER ADDRESS BOOK /-60 
USER PERSONAL SPACE 



E-MAIL CLIENT 
VIEWER\ 



67 



FIG. 1 



SUBSTITUTE SHEET (RULE 26) 



WO 01/77796 



PCT/US01/11486 



2/9 




SUBSTITUTE SHEET (RULE 26) 



WO 01/77796 



PCT/US01/11486 



3/9 



71 /-73 /-76 S-80/-82 /-84 r 86 r88 r90 



View Email Fax Convert New Move Rename Upload Dnload Delete 




Q Business notes 
O Bank check notes 

6 July 2000 6-8-2000 
— H E-mail to gates.NSB 6-8-2000 



-Q 16 July 2000-2.NSB 6-8-2000 



— Q 06 July 2000-3.NSB 6-8-2000 
m Q 08 July 2000 



FIG. 5 



SUBSTITUTE SHEET (RULE 26) 



WO 01/77796 



PCT/US01/11486 



4/9 



lUPG1-Phone.com QBE 



File info Edit Settings Help 



Go|ip://63.222.190.197/uplQa(|/defauft.as|a 



phorie.com 



Welcome to Digital 
Ink WAP Service J 
1t> Login 
2 register 



OK 




OO 



J. 



lUPG1-Phone.com QBE 



File info Edit Settings Help 



Go|IHtp://63.222.190.197/ypload/detaull.asB| 



•••• 
•••• 



phone.com 



User Id? 




gregl 




OK 


alpha 




J. 



FIG. 6 



FIG. 7 



SUBSTITUTE SHEET (RULE 26) 



WO 01/77796 



PCT/US01/11486 



5/9 



lUPG1-Phone.com EISE 



File Info Edit Settings Help 



G o |tittp://63.222. 1 90.1 97/uptoacydefauft.as 



phorie.com 



Password ? 
**** 




IUPG1-Phone.com PBS 



File jnfo Edit Settings Help 



GolHttp://63.222.t90.197/yploa[|/(lelaylt.as|a 



•••• 



phone.com 



Hello, Greg 

Pogrebinsky! 

1l>lnbox 

2 Send Email 

3 Send Last Page 

4 Send Last Page to 



OK 




OO 



J. 



FIG. 8 



FIG. 9 



SUBSTITUTE SHEET (RULE 26) 



WO 01/77796 



PCT/US01/11486 



6/9 



IUPG1-Phone.com QBE 



File Info Edit Settings Help 



phorie.com 



Folders/Documents 
1l>Explore 02-07-2001 

2 Explore 02-09-2001 

3 Explore 02-1 1-2001 

4 Explore 02-12-2001 

5 Explore 02-13-2001 



OK 




J. 



FIG. 10 



1 UPG1 -Phone.com QBE 



File info Edit Settings Help 



Go h 



#••• 
•••• 



phone.com 



Folders/Documents 
1t>Send 10_11_36.bmp 

2 Send10_12_03.bmp 

3 Up Level 

4 Home 



OK 



|send] |(C< 




JL 



FIG. 11 



SUBSTITUTE SHEET (RULE 26) 



WO 01/77796 



PCT/USO 1/11486 




FIG. 13 

SUBSTITUTE SHEET (RULE 26) 



WO 01/77796 



PCT/US01/11486 



8/9 



210 



oooop&o 

0000H 
00000 Qb 



208 



k 



212- 



rrr\ 

I 



\3 



216-\ 



274- 



I. <— .K 



1 



r 



217 



© 



/ 



-278 



FIG. 14 



SUBSTITUTE SHEET (RULE 26) 



WO 01/77796 



PCT/US01/11486 



9/9 




210 



208 



v 



230 




FIG. 15 



SUBSTITUTE SHEET (RULE 26) 



(12) INTERNATIONAL APPLICATION PUBLISHED UNDER THE PATENT COOPERATION TREATY (PCT) 



(19) World Intellectual Property Organization 

International Bureau 

(43) International Publication Date 
18 October 2001 (18.10.2001) 




PCT 



(10) International Publication Number 

WO 01/077796 A3 



(51) International Patent Classification 7 : 

G07C 9/00, G06F 3/033 



G06F 1/00, 



(21) International Application Number: PCT/US0 1/1 1486 

(22) International Filing Date: 9 April 2001 (09.04.2001) 

(25) Filing Language: English 

(26) Publication Language: English 



(30) Priority Data: 

60/195,491 



10 April 2000 (10.04.2000) US 



(71) Applicant (for all designated States except US): DIGITAL 
INK, INC. [US/US]; 70 Hastings Street, Wellesley, MA 
02481 (US). 

(72) Inventors; and 

(75) Inventors/Applicants (for US only): SCHILLER, Ilya 



[US/US]; 58 University Road, Brookline, MA 02445 (US). 
PITTEL, Arkady [US/US]; 101 Winthrop Road, Unit 1, 
Brookline, MA 02445 (US). SEGAL, Alec [US/US]; 812 
Memorial Drive, Apt. 409, Cambridge, MA 02139 (US). 

(74) Agent: FEIG ENBAUM, David, L.; Fish & Richardson 
P.C., 225 Franklin Street, Boston, MA 02110-2804 (US). 

(81) Designated States (national): AE, AG, AL, AM, AT, AU, 

AZ, BA, BB, BG, BR, BY, BZ, CA, CH, CN, CO, CR, CU, 
CZ, DE, DK, DM, DZ, EE, ES, FT, GB, GD, GE, GH, GM, 
HR, HU, ID, IL, IN, IS, JP, KE, KG, KP, KR, KZ, LC, LK, 
LR, LS, LT, LU, LV, MA, MD, MG, MK, MN, MW, MX, 
MZ, NO, NZ, PL, PT, RO, RU, SD, SE, SG, SI, SK, SL, 
TJ, TM, TR, TT, TZ, UA, UG, US, UZ, VN, YU, ZA, ZW. 

(84) Designated States (regional): ARIPO patent (GH, GM, 
KE, LS, MW, MZ, SD, SL, SZ, TZ, UG, ZW), Eurasian 
patent (AM, AZ, BY, KG, KZ, MD, RU, TJ, TM), European 

[Continued on next page ] 



(54) Title: USING HANDWRITTEN INFORMATION 



O 




-HANDWRITING DATA(C0NTENT) 
-DESTINATION (IP) ADDRESS 
^SENDER ID 
^ DELIVERY TYPE 
-PEN SERIAL NUMBER 



O ( INTERNET J) O 
13 



ADDRESS BOOK / 

SERVER ADDRESS 

HANDWRITING^ 
DATA CONTENT 



COMPRESSED' 
DATA CONTENT 



^33 
11 



44 



25 




ADDRESS BOOK 
BROWSER 
VIEWERS 40 



-41 



USER ID-PEN SERIAL NUMBER MAP 
fySER ACCOUNT aR 
USER MAILBOX^ y-61 
USER ADDRESS BOOK ^60 
USER PERSONAL SPACE 



E-MAIL CUENK 



VIEWER 



67 



41 



(57) Abstract: Handwriting 
data is received electronically 
from a remote user at a 
handwritten-information server, 
and the handwriting data is 
processed in accordance with 
instructions provided to the 
server by the user, e.g. by 
forwarding the handwriting 
data to a specified destination 
identified by the user. 



WO 01/077796 A3 I lllll llllllll II llllll lllll Nil I II III lllll lllll lllll lllll lllll llll lllllll llll llll llll 



patent (AT, BE, CH, CY, DE, DK, ES, FI, FR, GB, GR, IE, 
IT, LU, MC, NL, PT, SE, TR), OAPI patent (BF, BJ, CF, 
CG, CI, CM, GA, GN, GW, ML, MR, NE, SN, TD, TG). 

Published: 

— with international search report 



(88) Date of publication of the international search report: 

23 January 2003 

For two-letter codes and other abbreviations, refer to the "Guid- 
ance Notes on Codes and Abbreviations" appearing at the begin- 
ning of each regular issue of the PCT Gazette. 



INTERNATIONAL SEARCH REPORT 



I ationa) Application No 

FTT/US 01/11486 



A. CLASSIFICATION OF SUBJECT MATTER , 

IPC 7 G06F1/00 G07C9/00 



G06F3/033 



According to International Patent Classification (IPC) or to both national classification and IPC 



B. FIELDS SEARCHED 



Minimum documentation searched (classification system followed by classification symbols) 

IPC 7 G06F G07F 



Documentation searched other than minimum documentation to the extent that such documents are included in the fields searched 
Electronic data base consulted during the international search (name of data base and, where practical, search terms used) 

EPO-Internal 



C. DOCUMENTS CONSIDERED TO BE RELEVANT 



Category ° Citation of document, with indication, where appropriate, of the relevant passages 



Relevant to claim No. 



EP 0 865 192 A (CASIO COMPUTER CO LTD) 
16 September 1998 (1998-09-16) 

abstract 



column 


2, 


1 ine 


40 


-col umn 


3, 


line 


46 


col umn 


5, 


1 ine 


48 


-col umn 


6, 


1 ine 


23 


column 


6, 


1 ine 


46 


-col umn 


7, 


1 ine 


30 


col umn 


7, 


1 ine 


54 


-col umn 


8, 


1 ine 


5 


column 


8, 


1 ine 


47 


-col umn 


9, 


line 


4 



1,5, 

7-10, 

12-17,26 

2-4,6,18 



column 20, line 43 -column 21, line 40 
column 22, line 16 - line 33 
figures 1,2,23 



Further documents are listed in the continuation of box C. 



Patent family members are listed In annex. 



• Special categories of cited documents : 

'A' document defining the general state of the art which is not 
considered to be of particular relevance 
earlier document but published on or after the international 
filing date 

"L" document which may throw doubts on priority claim(s) or 
which is cited to establish the publication date of another 
citation or other special reason (as specified) 

"O" document referring to an oral disclosure, use, exhibition or 
other means 

"P" document published prior to the international filing date but 
later than the priority date claimed 



■T" later document published after the international filing date 
or priority date and not in conflict with the application but 
cited to understand the principle or theory underlying the 
invention 

■X" document of particular relevance; the claimed invention 
cannot be considered novel or cannot be considered to 
involve an inventive step when the document is taken alone 

'Y" document of particular relevance; the claimed invention 

cannot be considered to involve an inventive step when the 
document is combined with one or more other such docu- 
ments, such combination being obvious to a person skilled 
in the art. 

document member of the same patent family 



Date of the actual completion of the international search 



3 October 2002 



Date of mailing of the international search report 



10/10/2002 



Name and mailing address of the ISA 

European Patent Office, P.B. 5618 Patentlaan 2 
NL-2280 HV Rijswijk 
Tel. (+31-70) 340-2040, Tx. 31 651 epo ni, 
Fax: (+31-70) 340-3016 



Authorized officer 



Arbutina, L 



Form PCT/ISA/210 (second sheet) (July 1992) 



INTERNATIONAL SEARCH REPORT 


1 tional Application No 

PUT/US 01/11486 


C(Continuation) DOCUMENTS CONSIDERED TO BE RELEVANT 


Category ° 


Citation of document, with indication ,where appropriate, of the relevant passages 


Relevant to claim No. 


X 
Y 


US 5 612 720 A (ITO JUN ET AL) 
18 March 1997 (1997-03-18) 
column 6, line 36 -column 7, line 8 
column 12, line 34 - line 56 
column 13, line 25 - line 30 
column 14, line 41 -column 15 } line 20 
figures 1,76,10 




19-25 
27,31 


x 

Y 


WO 97 16799 A (GENOSSAR 0MRY ; BARON EHUD 

(IL); BARON TECH LTD (IL)) 

9 May 1997 (1997-05-09) 

page 8, line 22 -page 9, line 15 

page 10, paragraph 4 
page 11, last line -page 12, line 7 
page 18, line 28 -page 20, line 30 
figure 1 




32,33 

2-4,6, 
27-31 


X 


US 5 546 538 A (POLLACK FREDERICK J ET 
AL) 13 August 1996 (1996-08-13) 
abstract 

column 4, line 55 -column 7, line 15 
figures 2,3 




11,32,34 


X 
A 


NARAYANASWAMY S ET AL: "USING DATA ON 
DIGITAL CELLULAR AND PCS VOICE NETWORKS" 
BELL LABS TECHNICAL JOURNAL, WILEY, CA, 
US, 

vol. 3, no. 2, 1 April 1998 (1998-04-01), 
pages 58-75, XP000772944 
ISSN: 1089-7089 

page 67, right-hand column, paragraph 4 
-page 69, right-hand column, paragraph 2 
page 71, right-hand column, paragraph 2 
figures 2,3,5 




32-36 
19 


Y 


EP 0 717 368 A (SYMBIOS LOGIC INC) 
19 June 1996 (1996-06-19) 
column 2, line 45 - line 50 
column 5, line 11 - line 40 
figure 1 




28-30 


Y 


EP O 869 690 A (LUCENT TECHNOLOGIES INC) 
7 October 1998 (1998-10-07) 
column 1, line 24 - line 45 
column 2, line 17 -column 3, line 4 
figure 1 




18 


A 


US 5 805 719 A (HOFFMAN NED ET AL) 
8 September 1998 (1998-09-08) 
column 4, line 8 - line 22 
column 12, line 28 -column 13, line 24 
column 27, line 24 - line 28 
column 30, line 25 -column 31, line 10 
figure 1 




20,24,25 



Form PCT/ISA/210 (continuation of second sheet) (July 1992) 



International Application No. PCTXlS 01 A1486 

FURTHER INFORMATION CONTINUED FROM PCT/ISA/ 210 

This International Searching Authority found multiple (groups of) 
inventions in this international application, as follows: 

1. Claims: 1-17,26 

Processing hadwriting data in accordance with instructions 
provided by the user 

2. Claim : 18 

Providing device display capabilities independence for the 
data that is to be delivered 

3. Claims: 19-25,27,31-36 

Use of a mobile handwriting device for authenticating a user 

4. Claims: 28-30 

Generating visible qualities with handwriting device 



INTERNATIONAL SEARCH REPORT 



ernational application No. 
PCT/US 01/11486 



Box I Observations where certain claims were found unsearchable (Continuation of item 1 of first sheet) 

This International Search Report has not been established in respect of certain claims under Article 17(2)(a) for the following reasons: 
1 . | | Claims Nos.: 

because they relate to subject matter not required to be searched by this Authority, namely: 



I. | | Claims Nos.: 

because they relate to parts of the International Application that do not comply with the prescribed requirements to such 
an extent that no meaningful International Search can be carried out, specifically: 



3. | | Claims Nos.: 

because they are dependent claims and are not drafted in accordance with the second and third sentences of Rule 6.4(a). 

Box II Observations where unity of invention is lacking (Continuation of item 2 of first sheet) 

This International Searching Authority found multiple inventions in this international application, as follows: 

see additional sheet 



1 . I I As all required additional search fees were timely paid by the applicant, this International Search Report covers all 
1 1 searchable claims. 



As all searchable claims could be searched without effort justifying an additional fee, this Authority did not invite payment 
of any additiona! fee. 



3. 1 I As only some of the required additional search fees were timely paid by the applicant, this International Search Report 
1 ' covers only those claims for which fees were paid, specifically claims Nos.: 



4. | | No required additional search fees were timely paid by the applicant. Consequently, this International Search Report is 
restricted to the invention first mentioned in the claims; it is covered by claims Nos.: 



Remark on Protest | j The additional search fees were accompanied by the applicant's protest. 

| j No protest accompanied the payment of additional search fees. 



Form PCT/ISA/210 (continuation of first sheet (1)) (July 1998) 



INTERNATIONAL SEARCH REPORT 

Information on patent family members 


1 itional Application No 

PCI/US 01/11486 


Patent document 
cited in search report 


Publication 
date 


Patent family 
member(s) 


Publication 
date 



EP 0865192 A 16-09-1998 OP 10254989 A 25-09-1998 

JP 10260916 A 29-09-1998 

EP 0865192 A2 16-09-1998 

US 6366698 Bl 02-04-2002 



US 5612720 



WO 9716799 



US 5546538 



EP 0717368 



EP 0869690 



US 5805719 



18- 


-03- 


■1997 


JP 


3176743 


B2 


18-06-2001 








JP 


6214704 


A 


05-08-1994 








JP 


6230885 


A 


19-08-1994 








US 


5453762 


A 


26-09-1995 


09- 


-05- 


■1997 


AU 


7330596 


A 


22-05-1997 








WO 


9716799 


A2 


09-05-1997 


13- 


-08- 


■1996 


NONE 








19- 


-06- 


■1996 


DE 


69523024 


Dl 


08-11-2001 








DE 


69523024 


T2 


14-03-2002 








EP 


0717368 


Al 


19-06-1996 








OP 


8234902 


A 


13-09-1996 








US 


5883338 


A 


16-03-1999 


07- 


-10- 


1998 


US 


6393296 


Bl 


21-05-2002 








EP 


0869690 


A2 


07-10-1998 








JP 


10313340 


A 


24-11-1998 


08- 


09- 


1998 


US 


5613012 


A 


18-03-1997 








US 


5615277 


A 


25-03-1997 








AU 


5922696 


A 


29-11-1996 








BR 


9608580 


A 


05-01-1999 








CA 


2221321 


Al 


21-11-1996 








CN 


1191027 


A 


19-08-1998 








EP 


0912959 


Al 


06-05-1999 








JP 


11511882 


T 


12-10-1999 








WO 


9636934 


Al 


21-11-1996 








US 


2002111917 


Al 


15-08-2002 








US 


6366682 


Bl 


02-04-2002 








US 


6269348 


Bl 


31-07-2001 








US 


6230148 


Bl 


08-05-2001 








US 


6192142 


Bl 


20-02-2001 








US 


6012039 


A 


04-01-2000 








US 


6154879 


A 


28-11-2000 








US 


6397198 


Bl 


28-05-2002 








US 


5838812 


A 


17-11-1998 








US 


5870723 


A 


09-02-1999 








us 


5764789 


A 


09-06-1998 








us 


2001000535 


Al 


26-04-2001 








us 


5802199 


A 


01-09-1998 








us 


2001029493 


Al 


11-10-2001 








us 


2001039533 


Al 


08-11-2001 



Form PCT/ISA/210 (patent family annex) (July 1992) 



